Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable GDS and Support Runtime Context (__enter__, __exit__) for CuFileDriver and CuImage #106

Merged
merged 4 commits into from
Sep 29, 2021
Merged

Enable GDS and Support Runtime Context (__enter__, __exit__) for CuFileDriver and CuImage #106

merged 4 commits into from
Sep 29, 2021

Conversation

gigony
Copy link
Contributor

@gigony gigony commented Sep 28, 2021 •
edited

  • Enable GDS with GDS v1.0.0
    • Re-enable GDS with public package
    • Found an issue with a GDS static library and working on WSL
      1. Segmentation fault error is happening if we link GDS static library because GDS static library file (libcufile.a) available does not support non-CXX11 ABI (-D_GLIBCXX_USE_CXX11_ABI=0). Python wheel with manylinux2014 needs to be compiled with non-CXX ABI
        • Workaround the static library issue by using a shared library (if exists on the system) instead.
      2. Loading/using GDS shared library(libcufile.so) causes Assertion failure, file index :cufio-udev line :143 error on WSL
        • Workaround the WSL issue by not loading GDS shared library (calling cuFileDriverOpen()) if the platform is WSL.
  • Do not use GDS if in WSL
  • Add __enter__ and __exit__ for CuFileDriver
  • Add __enter__ and __exit__ for CuImage

Resolves #3

@gigony gigony added the feature request New feature or request label Sep 28, 2021
@gigony gigony added this to the v21.10 milestone Sep 28, 2021
@gigony gigony self-assigned this Sep 28, 2021
@gigony gigony requested review from a team as code owners September 28, 2021 19:36
@gigony gigony added this to PR-Needs review in v21.10 Release via automation Sep 28, 2021
@gigony gigony added the non-breaking Introduces a non-breaking change label Sep 28, 2021
@gigony
Copy link
Contributor Author

gigony commented Sep 28, 2021

CI/CD issue:
https://gpuci.gpuopenanalytics.com/job/rapidsai/job/gpuci/job/cucim/job/prb/job/cucim-cpu-cuda-build/CUDA=11.0/56/console

Line 958:

12:48:16     raise CondaMultiError(exceptions)
12:48:16 conda.CondaMultiError: Downloaded bytes did not match Content-Length
12:48:16   url: https://conda.anaconda.org/conda-forge/linux-64/cudatoolkit-11.0.3-h15472ef_8.tar.bz2
12:48:16   target_path: /opt/conda/envs/rapids/pkgs/cudatoolkit-11.0.3-h15472ef_8.tar.bz2
12:48:16   Content-Length: 998168165
12:48:16   downloaded bytes: 232106833

@gigony
Enable GDS with GDS v1.0.0
7f84ed8
@gigony
Do not use GDS if in WSL
d01cf13 
GDS v1.0.0 does not support WSL and executing
this can cause the following error:

    Assertion failure, file index :cufio-udev  line :143

So do not call cuFileDriverOpen() if the
current platform is WSL.
@gigony
Add __enter__ and __exit__ for CuFileDriver
edb624d
@gigony
Add __enter__ and __exit__ for CuImage
2d80efe
jakirkham
jakirkham reviewed Sep 29, 2021
View reviewed changes
Copy link
Member

@jakirkham jakirkham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Gigon! 😄 Looks good

So are we going to include this in Conda and Wheels somehow?

@gigony
Copy link
Contributor Author

gigony commented Sep 29, 2021

So are we going to include this in Conda and Wheels somehow?

yes, We want to include this patch in v21.10.00

@jakirkham
Copy link
Member

Sorry what I mean is how are the GDS libraries included in the packages? Or is this up to user to install somewhere as well? If the latter, where should they be installing the GDS libraries and what happens if they are not there?

@gigony
Copy link
Contributor Author

gigony commented Sep 29, 2021 •
edited

Sorry what I mean is how are the GDS libraries included in the packages? Or is this up to user to install somewhere as well? If the latter, where should they be installing the GDS libraries and what happens if they are not there?

@jakirkham
It's up to user to install GDS libraries (GDS libraries are available through CUDA Toolkit >=11.x . I need to check from which version libcufile is included but I can confirm that CUDA Toolkit 11.4 include it), or can install manually by following the instruction https://docs.nvidia.com/gpudirect-storage/troubleshooting-guide/index.html#gds-installing

If GDS library is not available, the filesystem API still works with compatibility mode (our own implementation to do the same behavior).

@jakirkham
Copy link
Member

Ah ok it's in the CUDA Toolkit. That was all I needed to know. Thanks Gigon! 😄

v21.10 Release automation moved this from PR-Needs review to PR-Reviewer approved Sep 29, 2021
@jakirkham
Copy link
Member

@gpucibot merge

@rapids-bot rapids-bot bot merged commit 68f6042 into rapidsai:branch-21.10 Sep 29, 2021
v21.10 Release automation moved this from PR-Reviewer approved to Done Sep 29, 2021
@gigony gigony mentioned this pull request Sep 29, 2021
Merged
@gigony gigony mentioned this pull request Feb 23, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jakirkham jakirkham jakirkham approved these changes

@quasiben quasiben Awaiting requested review from quasiben

Assignees

@gigony gigony

Labels
feature request New feature or request non-breaking Introduces a non-breaking change
Projects
No open projects
v21.10 Release
Done
Milestone
v21.10
Development

Successfully merging this pull request may close these issues.

Enable GDS with public package for v0.19.0
2 participants
@gigony @jakirkham